Non-Adversarial Learning: Vector-Quantized Common Latent Space for Multi-Sequence MRI

🌈 Abstract

The paper proposes a generative model called VQ-Seq2Seq that leverages intermediate sequences to estimate a common latent space among multi-sequence MRI, enabling the reconstruction of distinct sequences from this common latent space. The key advantages are:

• Unsupervised synthesis without adversarial learning
• Anti-interference ability to eliminate the effects of noise, bias fields, and artifacts
• Solid semantic representation ability with potential for one-shot segmentation

🙋 Q&A

[01] Preliminary

1. What is VQ-VAE and how does it differ from VAE? VQ-VAE uses a discrete latent space instead of the continuous latent space of VAE. The discrete latent space captures more structured features while ignoring some irrelevant details like artifacts.

2. What is a dynamic model and how does it utilize intermediate sequences? Dynamic models combine different generation tasks in a single model, allowing them to leverage intermediate sequences. For example, if there are paired T1-T2 and T2-Flair samples, a dynamic model can use T2 as an intermediate sequence to translate between T1 and Flair without direct paired samples.

[02] VQ-Seq2Seq

1. How does VQ-Seq2Seq establish the VQC latent space?

• VQ-Seq2Seq extracts continuous and discrete latent representations from input images using VQ-VAE.
• It then estimates the uncertainty of the discrete latent space across different sequences using statistics.
• Finally, it models the VQC latent space as a Gaussian distribution based on the estimated uncertainty.

2. How does VQ-Seq2Seq leverage the VQC latent space for generation? VQ-Seq2Seq uses a dynamic decoder to generate target sequences directly from the VQC latent space, without requiring adversarial learning.

3. What loss functions does VQ-Seq2Seq use?

• Pixel-level reconstruction loss (L1, SSIM, perceptual)
• Latent space consistency loss (MSE, contrastive)
• Total loss combining the above

4. How does random domain augmentation improve VQ-Seq2Seq? Random domain augmentation, including intensity transformations, cross-sequence translation, and random domain translation, improves the stability of VQ-Seq2Seq and the anti-interference ability of the VQC latent space.

[03] Experiments

1. How does VQ-Seq2Seq perform compared to other methods on cross-sequence generation tasks? VQ-Seq2Seq outperforms other GAN-based methods like MM-GAN and ResViT, especially on unpaired generation tasks (T1->T2, T1->Flair). It can perform single-step generation without the information loss and error accumulation of multi-step generation.

2. What are the benefits of the VQC latent space in terms of anti-interference and representation ability? The VQC latent space shows strong anti-interference ability, effectively preventing the impact of noise and bias fields on image reconstruction. It also demonstrates excellent representation ability, enabling a one-shot brain tumor segmentation model to outperform a model trained on the original images.